Object Condition Sensing

ABSTRACT

Embodiments are described for multi-sensor systems for real-time embedded monitoring of object senses and mixed-mode object conditions. In one or more embodiments, various sensors can separately provide disparate analog signals representing different measurable attributes regarding a sensed object. For example, sensors may separately sense temperature, pressure, or other biometric values. In some embodiments and according to specified rule sets or other qualifying parameters, a digital signal can be generated by a processor and/or controller to indicate one or more conditions of the sensed object according to sensor input values. Additionally or alternatively, a multi-sensor scheme may be coupled to a digital network and/or coupled thereto for simulation and/or communication applications.

RELATED APPLICATIONS

This application is a continuation of and claims priority to U.S. patentapplication Ser. No. 11/058,780, entitled “Method and Apparatus forMulti-Sensor Processing,” filed on Feb. 15, 2005, which in turn is acontinuation of and claims priority to U.S. patent application Ser. No.09/949,257, entitled “Method and Apparatus for Multi-Sensor Processing,”filed on Sep. 7, 2001 and now issued as U.S. Pat. No. 6,922,664, whichin turn is a divisional of and claims priority to U.S. patentapplication Ser. No. 09/220,784, entitled “Method and Apparatus forMulti-Sensor Processing,” filed on Dec. 23, 1998 and now issued as U.S.Pat. No. 6,415,188, the disclosures of which are incorporated in theirentirety by reference herein.

BACKGROUND

Conventional electronic sensors are used in various industrial andcommercial applications, for example, whereby certain transducer-typedevice may measure a physical condition and generate an electricalsignal which represents such measured condition. Conventional sensors,however, typically generate analog signals and are not designed tointerface easily to digital networks. Although more recently, electronicindustry attention has increasingly turned toward coupling so-calledembedded processing elements to digital networks, such recent approachesprovide limited capability in processing multi-sensor systems,particularly for digital networks.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is general system and network diagram according to one or moreembodiments.

FIG. 2 is block diagram of a client according to one or moreembodiments.

FIG. 3 is flow chart of steps in a method according to one or moreembodiments.

FIG. 4 is a block diagram of biometric and/or multi-sensor modulesaccording to one or more embodiments.

FIG. 5A is a diagram of a simulator module according to one or moreembodiments.

FIG. 5B is a diagram of a communication module according to one or moreembodiments.

DETAILED DESCRIPTION

Embodiments discussed herein include multi-sensor systems and methodsthat can enable interactive sensing of mix-signal attributes todetermine one or more object conditions. Sensors can separately measuredifferent physical attributes to generate corresponding analog signals.According to one or more specified rule sets and/or other qualifyingparameters, a digital signal can be generated by a processor and/orcontroller to represent one or more conditions of the sensed objectaccording to such sensor signals. According to some embodiments,multi-sensor schemes may be coupled to a digital network or electronicfacility for simulation and/or communication.

FIG. 1 illustrates a general diagram of a digital system having aninterconnected network 10 configured to couple servers 12, 14, clients18, 20, a storage repository 16, and a controller 22. The network 10 mayinclude one or more local, medium or wide area interconnections or otherdigital wired or wireless linkages accessible according to one or morestandard networking protocols, such as the Internet, World-Wide Web,TCP/IP, or other Internet Protocol (IP) convention. The clients 18,20,servers 12, 14, controller 22, and storage 16 may include one or morenetwork-accessible computers, processors, controllers or other systemnodes for processing and/or storing digital data.

According to some embodiments, the server 14 can serve as afault-tolerant functional mirror or data replication facility for server12, such that servers 12, 14 can be managed to store some or all of thesame data. Additionally or alternatively, the client 18 can serve as apeer of the client 20, such that client-to-client communication may beaccomplished for direct data or signal transfer therebetween. Accordingto some embodiments, the storage repository 16 can serve as one or morenetwork-accessible peripheral storage or memory facilities for storingdigital data, such as the temporary caching of simulation data,communications data, control files, and/or signals.

In some embodiments, the controller or monitor 22 can serve as one ormore network-accessible computing or processing facilities for enablingsensing and/or related functions or other network system managementtasks, according to one or more embodiments described herein. Forexample, the controller 22 may serve as a system manager forinitializing, coordinating, and/or controlling network tasks or otherclient-server distributed applications, such as video-conferencing orsimulation programs executed among a number of client users coupled tothe network.

FIG. 2 illustrates a block diagram of the client 20 according to one ormore embodiments. In this particular example, the client 20 includes avideo or screen display 32, a network interface 24, a peripheral device26, a storage or memory 28, a processor 30, a speaker or audio device46, an actuator or mechanical device 48, a sensor or biometric device50, a dispenser device 52, a keyboard or mouse device 54, a camera 42,and a microphone 44. The display 32 may include a graphics-based usermenu interface 34 and one or more symbolic, simulated, video, animated,or otherwise graphics-based depictions of individual or objects 38having an identifiable face 36, icon, avatar, or other representationthereof.

In some embodiments, the display 32 may provide visual informationaccording to holographic, 3-dimensional, virtual reality, or othersimilarly enhanced graphic dimensional effects. The speaker or audiodevice 46 may provide multi-channel or other enhanced stereoscopic or“surround-sound” effects. Additionally or alternatively, the mechanicaldevice 48 may operate as a micro or miniaturized actuator, a roboticlink, a vibrator, or other movable element. The dispenser device 52 mayelectro-mechanically provide a client user with requested, programmed,or otherwise computer-assisted packaged goods, medicine, liquids,solutions, consumable items, and/or other dispensable material.

In some embodiments, the dispenser device 52 may indicate to the server12, 14 or other network node one or more current conditions related todispensing material, such as a remaining amount. Optionally oradditionally, one or more sensors may be implemented on the keyboard 54or mouse device, such that user finger or hand condition sensing can befacilitated.

It is contemplated herein that client 20 may be configured, at least inpart, by assembling a conventional personal computer, a TV set-topdevice, a laptop, a palmtop, an engineering workstation, acomputer-implemented automated transaction booth or “kiosk”, and/orother network-accessible processing node, which is programmed andequipped to function according to one or more embodiments describedherein. In some embodiments of the present invention, it is furthercontemplated that the biometric device or sensor array module 50 may becoupled directly to the network 10, without being included in, or havingto couple through, the client 20.

FIG. 3 is a flow chart illustrating steps in a method for implementing amulti-sensor system for real-time embedded monitoring of one or moreobjects under mixed-mode sensing conditions in accordance with one ormore embodiments. Further, in this regard, as shown in the block diagramof FIG. 4, the biometric sensing device 50 can include a networkinterface 80 and/or various sensors 88 for separately providing, throughanalog-to-digital (AID) converter circuit, to a processor 84 and astorage 86 of a sensor array 82 disparate analog signals representingdifferent measurable attributes regarding sensed object.

According to some embodiments, sensor subsystem integration can beachieved through a microelectromechanical systems (MEMS) approach byproviding most or all electronic circuits, including the processor 84,the storage 86, the interface 80, the sensors 88, and/or any A/Dconverter circuits on a common semiconductor substrate or die, althoughthe interface 80 and/or the sensors 88 can be provided on separatesubstrates or dice. In some embodiments, it is contemplated thatmultiple sensors may be coupled and provide mix-mode sensed signals to acommon processing circuit.

In some example embodiments, one or more sensor modules may beimplemented, at least in part, for functional operation according tosome embodiments described herein, using commercially available devices,such as product part numbers EDI 520 (smart sensor module), EDM 710(sensor interface circuit), and/or RS-485 (network node) fromElectronics Development Corporation (Columbia, Md.).

According to some embodiments, one set of one or more sensors 88 maysense and monitor one type of object condition, and another set of oneor more sensors 88 may sense and monitor another type of objectcondition. For example, for a given individual subject being monitored,a first sensor set may monitor person temperature at one or more bodilysources, while a second sensor set may monitor same person perspirationrate at one or more bodily sources.

In some embodiments, such multi-sensor set system can monitor relatedand/or possibly unrelated conditions associated with a common object orobject set that is monitored during simultaneous, temporally close, orotherwise relatively proximate time periods. In accordance with someembodiments, the sensor array processing circuitry 82 can operate toreceive multi-sensor signals which indicate sensed conditionsrepresenting different sensor or sensing signal types, classes,attributes, and/or other monitorable grouping, and thus process sucheffectively mix-mode signals to determine whether certain monitoredobject(s) or individual(s) previously, currently, or is likely to, fallwithin certain specified condition(s), as determined by processing suchmix-mode sensor signals.

Some embodiments can utilize a mix-mode approach whereby signalprocessing by the processor 84 can be performed in an effectivelycombined and integrated manner according to one or more common rulesets, user specifications, and/or other programmed instructions, whichcan classify and indicate one or more monitored object conditions orsensed states logically or inferentially according to actual inputsensed signal values corresponding to different modes or otherphysically measured groupings.

In some embodiments, to achieve improved overall system or partialsubsystem integration, the processor 82 or functionally comparabledigital signal processing circuit can serve separately to receive andprocess multiple-type sensor signals or signal sets, whereupon suchsensor signals or signal sets represent different physically sensed orotherwise electronically monitorable conditions, states, attributes,modes, or quality of sensing thereof.

For example, the processor 84 may locally execute an instruction set inthe storage 86 to analyze, compare, correlate, and/or process receivedmix-mode signals according to specified rules or heuristics to indicateremotely whether subject individual may be diagnosed as having symptomsof one or more medical conditions, and therefore require dispensing ofcertain medicines or other goods or supplies. Such “smart sensor”processing and analysis may also be accomplished using a digital signalprocessor having logically or functionally equivalent programming and/orcircuit configuration.

Thus, according to some embodiments, such intelligently determinedconditions may be generated as findings, flags, warnings, or otherindications provided as feedback in a digital packet, datagram, frame,or other capsulized format through the interface 80 for network access,for example, to serve as input values to the simulator module 90 forfantasy gaming applications, or the communication module 94 forvideo-conferencing applications. Optionally or additionally, theinterface 80 may provide sensor feedback data signal through the network10 according to one or more established or known network or businterface standards, such as IEEE 1451 standard for interfacing to smartsensors.

In some embodiments, the sensors 88 may be fixed, mobile,wirelessly-connected or wired, and separately sense temperature,pressure, physiological vital information (e.g., heart beat rate, bloodpressure, etc.), and/or other biometric values. For example, one or moresensors in the array 82 may be worn, implanted, attached, or provided byindividual objects on clothing or vehicle, and/or provided in contactthereto with one or more external or internal bodily locations.

In some embodiments, it is contemplated herein that the sensors 88 maybe provided, for example, as one or more silicon-based micro-machinedmicrostructure cavities which may be implanted for applicable modes suchas neuro-electronic monitoring of cell metabolism and/or controlling ofcell activity.

According to one or more embodiments, the sensors 88 may be configuredto monitor one or more voluntary and/or involuntary conditions, such asdistinct sensory modes, of a subject user or other observed party, suchas skin temperature, perspiration rate, or other measurablephysiological conditions. In particular, the sensor array 82 can operatein an intelligent or “smart” manner, such that, for example, distributedsensors, actively or passively, synchronously or asynchronously, senseand generate sensing signals according to pre-programmed logical rulesand/or other user specifications, such as determining acceptablemanufacturing tolerance or safety conditions.

In some embodiments, the sensor array 82 can function selectively orlogically to screen, filter, censor, and/or exclude or enable access ofrepresentative signaling of certain sensed or otherwise observedconditions, such as during specified times, dates, or other specifiedtemporal segments, such as control of mature-audience programming. Forexample, within given monitoring period, the processor 84 may compute orcompare to determine, and accordingly indicate for network access, thatreceived sense signals comply or violate a certain specified range, orfall within particular margins. Optionally or additionally, theprocessor 84 may selectively access one or more of the sensors 88belonging to one or more selected modes, groupings, or other pre- oruser-specified classifications, such as higher-resolution, reliability,or quality sensor groups.

Additionally or alternatively, in some embodiments the presentdistributed sensor array architecture may provide for directed,hierarchical, self-navigating and/or organizing, adaptive, or flexiblyprogrammable access to one or more sensors in the array 82, such as byproviding tiered quality of service access to varying levels of sensorsensitivity, reliability, accuracy, performance, or other relevantsensor parameters.

In some hierarchical-style embodiments, a first set of mix-mode sensedsignals can be received for processing as described herein to generate afirst processed signal indicating one monitored mode or other level offunctional abstraction, which represents a logical determinationaccording to rule-based interpretations or analyses of the first set ofreceived mix-mode signals. According to some embodiments, a second setof mix-mode sensed signals can then be received for processing as wellto generate a second processed signal indicating another monitored modeor other level of functional abstraction, which represents a differentlogical determination according to rules-based interpretation and/oranalysis on such second set of received mix-mode signals.

According to some embodiments, such mix-mode first and second processedsignals can be received, in hierarchical or tiered fashion, for furtherprocessing according to rules-based interpretation and/or analysis asdescribed herein to generate a third processed signal to serve assensory feedback according to higher-level monitored mode and/or otherlevel of functional abstraction.

In some embodiments and according to specified rule set or otherqualifying parameters, a digital signal can be generated by theprocessor or controller 84 to indicate one or more conditions of thesensed object according to sensor input values. Additionally oralternatively, referring to FIGS. 5A and 5B, such multi-sensor schememay be coupled to the digital network 10 and/or coupled thereto forsimulation and/or communication applications 90, 94, as described inmore detail herein.

Referring to FIG. 3, initially, the multi-sensor system can beconfigured 56 functionally with system components as illustrated in FIG.2, and source or object software, computer program, or other instructioncode can be installed 58 in such system for operation as describedherein.

Optionally or additionally, one or more system users or correspondingclients 18, 20 can subscribe 60 to, or can be otherwise provided with,authorized user or group user accounts for secure system access, forexample, as a member for enabling exclusive access to one or morenetwork-accessible programs, files, or other restricted objects.Authorization may be accomplished by identifiable user entry and/orother input through a keyboard, a mouse, voice, facial imagerecognition, finger print detection, retinal scan, smart card input, orother unique user entry, for example, by using the peripheral device 26as an input processing device.

Optionally or additionally, authorization may be provided by user entryof a unique password or other identifiable signature, such as geneticsequencing or other related data. Also, optionally or additionally, uponuser authorization 60, an authorized user may cause client 20 to conductone or more comparison 62 of various objects available from a number ofsource nodes accessible through the network 10.

In some embodiments, object compare 62 can enable a user to conducton-line product catalog shopping and/or select one or more desiredobjects using a conventional network user interface, such as Internetbrowser application software. In this way, a user may specify 64 one ormore objects for searching and subsequent comparison 62 thereof,enabling desired objects to be found and/or identified for a subsequenttransaction. Furthermore, an identifiable user entry for authorizationpurposes can provide a server source with a tracking basis to bill orcredit a user account for service, as well as to monitor and/or recorduser usage history, behaviors and/or preferences.

In some embodiments, the server 12 or the storage 16 may serve asnetwork-accessible sources for requesting, searching, renting, buying,and/or down-loading various software components, upgrades, or other codeor data, such as text, graphics, audio, video, models, vectors, images,fantasy or sports games, instructions, commands, or other electronicallytransmittable messages or signals, which are sensed, user-selectedand/or programmed or monitored according to one or more embodimentsdiscussed herein.

According to some embodiments, the source server 12 and/or thecontroller 22 may monitor usage and/or license distribution, usage orcopying of such down-loaded software to certain target or requestingclients 20, 18. Additionally or alternatively, in a code distributionscheme, the network 10 can serve as a real-time or interactive channel,architectural interface, or transaction platform for enabling securesubscription by multiple users or clients, particularly for providingmulti-sensor related applications.

Optionally or additionally, in some embodiments a user may define 66 oneor more rules and/or other heuristic instruction sets according to oneor more high-level functional or programming languages or applicationprogramming interfaces, which may be applied as attributes and/orconditions 68 to a sensing scheme, as described herein. For example,applied attributes 68 may include user-selected object characteristics,mappable facial imaging features, and/or language translation dictionaryfor processing simulated or communicated applications data.

In some embodiments, multi-sensor functionality can be implemented inthe context of, or overlaid upon, simulation and/or communication 70,respectively using simulator module 90 and/or communication module 94,as shown in FIGS. 5A and 5B. For example, sensor functionality canprovide input/output sensed signals 92, 96, 98, whereupon certain sensorsignals can be generated 72 in response to detection and measurement ofphysical conditions or attributes.

Regarding deployment of simulator module 90, in some embodiments theclient 20, 18 can be provided with, and/or have access to, one or moresoftware and/or hardware-based simulation or emulation programs orfunctionalities for representing the logic, behavior, functionality orother simulatable attributes of a modeled design, operation, condition,prototype, component, circuit, environment, or othercomputer-representable entity.

In some example embodiments, the simulator module 90 may include one ormore commercially-available computer-implemented simulation programswhich operate using, at least in part, one or more simulation models.Thus, during simulation of such provided models, one or more inputvectors, data or other signals can be applicable thereto, such that thesimulator may compute and thereby generate one or more proper outputvectors, data or other signals responsively therefrom. Such outputsignals may cause one or more client output devices, such as the audiodevice 46, the mechanical device 48, the display 32, and/or thedispenser device 52 to function accordingly and/or interact responsivelywith client use.

In some example embodiments, the simulator module 90 may be embodied toprovide single and/or multiuser interactive gaming, therapy, and/orexercise functionality. In some embodiments, such simulationfunctionality can operate in response to, among other things,multi-sensor input signals to enhance a simulation experience, asgenerated according to one or more embodiments discussed herein.

According to some embodiments and regarding deployment of thecommunication module 94, the client 20, 18 can be provided with, or hasaccess to, one or more software and/or hardware-based communicationsprograms, functionalities, or other facilities for transmitting and/orreceiving communications signals for bidirectional or duplex signalinteraction between a number of network-accessible processors or othernodes therein. For example, the communication module 94 may be embodiedin a videoconferencing system configured between two or more networkedcomputers for effectively real-time exchange of images and/or live videobetween communicating clients or peer parties. In some embodiments, suchcommunication can functionality operate by transmitting and/orreceiving, among other things, multi-sensor signals to enhancecommunication experience, as generated according to one or moreembodiments discussed herein.

Further to some embodiments and in accordance with such sensor response,at block 74 the communication and/or simulation modules 94, 90 and/orprior user specification may be modified, corrected, and/or changed.Optionally or additionally, from time to time, client software and othersystem parameters may be updated, such that client and/or system codemay be remotely programmably upgraded or remapped 76.

In some embodiments discussed herein, a multi-sensor system is providedfor networked cooperation or feature overlay with simulators and/orvideo-conferencing applications, whereupon, for example, conventionalnetworked, interactive fantasy gaming programs and/or videoconferencingsystems are enabled with effectively enhanced input or physical sensingof user or other object associated therewith.

In some embodiments an overlaid approach can be utilized to enablevariously categorized sets of real-time sensory feedback to becollected, computed, and/or transmitted from one or more smart sensorarrays to provide additional advanced ways for improving networking andcontrol, and thereby raise the level and quality of electroniccommunication and general user interactivity.

Foregoing described embodiments are provided as illustrations anddescriptions. They are not intended to limit the scope of the claimedembodiments to the precise forms or applications described. Inparticular, it is contemplated that functional implementations of theembodiments described herein may be implemented in hardware, software,firmware, and/or other available functional components or buildingblocks for various sensor-related commercial, industrial, medical,educational, media, broadcast, entertainment, food, agriculture,clothing, retail, fashion, defense, military, aerospace, automotive,transport, shipping, construction, design, finance, biotech,manufacturing, electronic, security, communications, information, orother related applications, systems or implementations.

Other variations and embodiments are possible in light of the discussionherein, and it is thus intended that the scope of invention not belimited by this Detailed Description.

1. A sensor architecture comprising: a first sensor configured tomonitor a first physical attribute of an object and generate acorresponding first signal; a second sensor configured to monitor asecond physical attribute of the object and generate a correspondingsecond signal, wherein the first and second sensors are configured to becoupled as peers on a network; a sensor array configured to controlaccessibility of the first and second sensors and to process the firstsignal and the second signal to indicate one or more physiologicalconditions associated with the object; and an output device configuredto provide output based on the one or more physiological conditions. 2.The sensor architecture of claim 1, wherein the output is a videosignal.
 3. The sensor architecture of claim 1, wherein the output isconfigured to enable a client device to implement one or more of gaming,therapy, or exercise functionality.
 4. The sensor architecture of claim1, wherein one or more of the first signal or the second signal areconfigured to be utilized to diagnose one or more medical conditions. 5.The sensor architecture of claim 1, wherein the output device isconfigured to dispense one or more medications.
 6. The sensorarchitecture of claim 1, wherein the output device comprises one or moreof: a display device; an audio device; a dispenser device; or amechanical device.
 7. The sensor architecture of claim 1, wherein one ormore of the first physical attribute or the second physical attributecomprises one or more of: temperature; perspiration rate; or bloodpressure.
 8. An apparatus comprising: a first sensor configured tomonitor a first physical condition of an object and generate acorresponding first signal according to a first rule; a second sensorconfigured to monitor a second physical condition of the object andgenerate a corresponding second signal according to a second rule; and astorage configured to provide the apparatus with one or more of thefirst rule or the second rule according to one or both of a usagelicense or subscription.
 9. The apparatus of claim 8, further comprisinga sensor array configured to process the first signal and the secondsignal to indicate one or more physiological conditions associated withthe object.
 10. The apparatus of claim 8, wherein one or more of thefirst signal or the second signal are configured to enable the apparatusto implement one or more of gaming, therapy, or exercise functionality.11. The apparatus of claim 8, wherein one or more of the first sensor orthe second sensor are configured to monitor one or more of the firstphysical condition or the second physical condition via contact with aperson.
 12. The apparatus of claim 8, wherein the output signal isconfigured to cause the apparatus to perform a function associated witha physiological condition indicated by one or more of the first physicalcondition or the second physical condition.
 13. The apparatus of claim12, wherein the apparatus comprises one or more of: a display device; anaudio device; a dispenser device; or a mechanical device.
 14. A tangiblecomputer-readable medium having stored thereon, computer-executableinstructions that, if executed by a computing device, cause thecomputing device to perform a method comprising: monitoring a firstphysical condition of an object with a first sensor to generate,according to a first rule, a corresponding first signal; monitoring asecond physical condition of the object with a second sensor togenerate, according to a second rule, an associated second signal,wherein the first sensor and second sensor are coupled as peers on anetwork, and wherein each of the first sensor and second sensor isaccessible on the network based on one or more sensor parameters; anddetermining a physiological condition of the object based on the firstsignal and the second signal.
 15. The tangible computer-readable mediumof claim 14, wherein the method further comprises: monitoring a thirdphysical condition of the object with a third sensor to generate,according to a third rule, a corresponding third signal, wherein thethird sensor is coupled to the first and second sensors as peers on thenetwork, and wherein the third sensor is accessible on the network basedon one or more of the sensor parameters; and determining thephysiological condition of the object based additionally on the thirdsignal.
 16. The tangible computer-readable medium of claim 14, whereinthe first sensor is configured to operate according to a high-levelprogramming language or application programming interface.
 17. Thetangible computer-readable medium of claim 14, wherein data associatedwith one or both of the first signal or second signal is accessible to aclient device.
 18. The tangible computer-readable medium of claim 17,wherein one or both of the first signal or the second signal areconfigured to cause the client device to implement one or more ofgaming, therapy, or exercise functionality.
 19. The tangiblecomputer-readable medium of claim 18, wherein one or both of the firstsignal or the second signal are configured to cause an output deviceassociated with the client device to perform an output functioncomprising at least one of: producing sound; causing a display;producing video; or dispensing a good.
 20. The tangiblecomputer-readable medium of claim 14, wherein at least one of the one ormore sensor parameters is associated with: sensor sensitivity; sensorreliability; sensor accuracy; or sensor performance.